The scientific method depends on conducting experiments and analyzing the data obtained from such experiments. Often, the data analysis involves statistical methods. When analyzing experimentally obtained data, it is often helpful to fit the experimentally obtained data to a curve. This process may be referred to as “curve fitting”. Curve fitting typically involves defining a “curve” (i.e. a function sometimes referred to as a “model”) that relates one data value to one or more other data values and selecting parameters of the curve such that the curve estimates the relationship between the data values. By way of non-limiting example, selection of the parameters of the curve may involve selection of coefficients of a polynomial function.
The curve established in the curve fitting process is typically an estimate, and not an exact representation, of the relationship between the known data values. The curve fitting process may involve the use of a cost function which provides a metric as to how closely the curve determined in the curve fitting process estimates the relationship between the data values. Curve fitting processes often seek to determine a curve which minimizes (or maximizes) the cost function. Minimization (or maximization) of the cost function may be referred to as “optimizing” the fitted curve.
One well known example of a cost function is the “least squares” cost function. The least squares cost function represents a sum of the squares of the residuals—i.e. a sum of the squares of the differences between values generated by the fitted curve and corresponding values from the data. Optimizing a fitted curve using a least squares cost function is a well known problem for which there are a number of known solutions. Examples of such solutions include the so called “Newton's method” and “gradient descent method”.
There are many drawbacks with the least squares cost function and the associated optimization methods. For example, iterative optimization is generally required to minimize the cost functions associated with non-linear models. Iterative optimization is typically performed using suitably programmed computer(s) or similar data processor(s). Using a least square cost function and the gradient descent method requires a relatively large number of iterations for the optimization to converge. Optimization techniques involving a large number of iterations are computationally expensive (i.e. requiring relatively large amounts of processing time and/or computational storage). Another drawback associated with least squares optimization processes is that they are relatively non-robust to outlying data values. Accordingly, if there is an obvious error with one or more of the data values (e.g. because a number was mistyped into a keyboard or a measurement apparatus was uncalibrated at the time of measurement), the resultant model obtained using least squares optimization could be skewed.
There is a desire to provide optimization methods for curve fitting which are relatively easy to perform and which ameliorate some of the drawbacks associated with prior art techniques.